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1  Abstract 


This  report  presents  a  summary  of  the  technical  status  for  the  period  8/1/95-10/31/95. 
A  major  milestone  was  passed  during  this  period  with  the  running  of  a  complete  neural 
network  training  on  our  SPERT  neurocomputing  board.  With  this  advancement,  our  multi¬ 
year  effort  to  apply  digital  VLSI  chip  design  to  speech  recognition  algorithm  development 
has  come  to  fruition.  In  addition,  our  effort  in  the  area  of  analog  auditory  preprocessors 
for  speech  recognition  continues  to  show  good  progress. 


2  Technical  Status 

2.1  The  TO  Processor  and  SPERT  board 

The  prototype  SPERT  boards  described  in  our  previous  status  report  are  in  daily  use,  pri¬ 
marily  for  speech  recognition  research.  The  next  run  of  30  boards  is  nearing  completion, 
and  will  greatly  enhance  our  computational  capacity.  Nearly  half  of  these  boards  are  com¬ 
mitted  to  collaborators  at  other  sites.  Before  taking  delivery  of  their  board,  each  site  will 
send  a  researcher  to  Berkeley  for  training  on  the  Torrent  architecture  and  SPERT  tools. 
By  the  end  of  this  quarter,  we  expect  to  have  an  active  international  community  of  Torrent 
users. 

With  more  than  six  months  of  experience  since  the  first  TO  die  was  tested,  we  are  pleased 
to  report  that  no  functional  errors  have  been  found  in  the  processor  design. 

2.2  Software 

At  the  beginning  of  the  quarter,  our  new  MLP  library,  ‘‘Quicknet”,  was  made  available 
to  speech  researchers  at  ICSI.  Quicknet  provides  a  high  level  interface  for  training  and 
running  neural  nets  on  either  SPERT  boards  or  workstations.  Several  programs  based 
upon  the  library  are  now  being  used  for  training  MLPs  for  speech  recognition  tasks. 

Additional  experiments  have  been  performed  to  determine  the  effects  of  using  fixed  point 
for  speech  recognition  training.  Our  latest  results  show  no  significant  differences  in  the  cross 
validation  error  rate  when  comparing  training  runs  using  floating  point  computation  on 
workstations  with  fixed  point  SPERT  boards.  There  is  a  significant  difference  in  training 
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Enclosed  you  will  find  the  quarterly  progress  report  for  the  research  project  for  Professor  John 
Wawrzynek:  “Construction  of  a  Connectionist  Network  Supercomputer,”  ONR  URI  Grant  No.: 
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time,  however,  with  the  SPERT  board  between  6X  and  20X  faster  than  a  SparcStation 
20  on  complete  training  runs,  including  the  overhead  of  initialization,  disk  I/O  and  data 
format  conversions. 

The  Torrent  fixed  point,  floating  point  and  integer  libraries  have  been  improved.  Con¬ 
siderable  time  was  spent  working  on  the  system  tools  and  enhancing  code  portability. 

There  was  also  significant  progress  in  the  area  of  high  level  programming  for  parallelism. 
Sather  release  1.0.8  is  the  first  to  include  an  extensive  Fortran  interface  which  significantly 
improves  the  usability  of  the  system  for  scientific  computation.  An  article  on  this  appeared 
in  the  September  1995  issue  of  Computers  in  Physics.  Our  group  was  also  awarded  a  grant 
from  Lawrence  Livermore  Natonal  Lab  supporting  use  of  their  Meiko  CS2  super  computer. 

The  Myrinet  system  for  parallel  Sather  has  been  installed  and  is  fuUy  functional.  This 
is  being  used  as  a  platform  for  language  and  algorithm  development  and  also  as  a  testbed 
for  possible  deployment  in  larger  CNS  systems.  A  first  parallel  version  of  the  ICSIM  con- 
nectionist  simulator  is  nearing  completion. 

2.3  Analog  VLSI  pre-processors. 

As  described  in  the  previous  progress  report,  we  have  been  applying  our  three-chip  sound 
pre-processing  system  to  a  speaker  independent  speech  recognition  problem.  We  are  con¬ 
ducting  our  experiments  using  a  200  speaker,  telephone  quality,  isolated  word  database  (1- 
9,  “oh”, “zero”,  “yes”,  “no”),  combining  our  pre-processing  system  with  a  Hidden  Markov 
Model  based  speech  recognizer  using  Multilayer  Perceptrons  for  phoneme  classification. 

In  the  last  quarter,  we  have  continued  to  improve  the  recognition  performance  on  the 
system,  achieving  a  4.1%  error  rate.  While  this  score  still  falls  short  of  the  best  software- 
based  front-ends  (1.8%  error,  using  the  same  database  and  testing  methods),  the  recognizer 
is  comparable  to  systems  current  in  use:  commercial  isolated-word  telephone-quality  digit 
recognizers  achieve  2-5%  error  performance  in  the  field,  handling  a  vocabulary  that  is 
slightly  smaller  vocabulary  (11  words)  than  our  13- word  task. 

Our  research  plans  for  future  work  on  the  recognition  system  require  major  changes  to 
our  experimental  framework,  including  switching  to  a  more  challenging  database,  and  the 
application  of  new  algorithms  for  speech  recognition.  Before  this  work  begins,  we  believe 
it  is  appropriate  to  disseminate  information  on  our  current  recognition  system,  including 
representation  choices,  feature  extraction  methods,  experimental  results,  and  formulation 
of  an  agenda  for  continued  research  into  auditory  models.  Much  of  the  work  this  quarter 
has  been  in  preparing  experimental  results  suitable  for  publication,  and  on  the  preliminary 
drafting  of  a  journal  paper  describing  the  work. 

Also  in  the  past  quarter,  we  received  word  of  the  acceptance  of  a  paper  we  submitted  to 
the  Neural  Information  Processing  Systems  conference  about  the  chip  design  work  for  our 
three-chip  sound  pre-processing  system.  We  are  now  preparing  our  spotlight  presentation 
and  poster  for  this  early  December  meeting,  as  well  as  an  8  page  paper  describing  the  chip 
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design  for  the  conference  proceedings. 


3  Recent  Publications 


Krste  Asanovic,  James  Beck,  Bertrand  frisson,  Brian  E.  D.  Kingsbury,  Nelson  Morgan  and 
John  Wawrzynek,  “The  TO  Vector  Microprocessor,”  Proceedings  of  Hot  Chips  VII,  August 
1995. 

Bourlard,  H.,  Konig,  Y.,  and  Morgan,  N.,  “REMAP:  Recursive  Estimation  and  Maxi¬ 
mization  of  a  Posteriori  Probabilities  in  Connectionist  Speech  Recognition,”  Proceedings  of 
Eurospeech  1995,  pp.  1663-1666,  Madrid,  Spain. 

Lazzaro,  J.  and  Wawrzynek,  J.,  “Silicon  Models  for  Auditory  Scene  Analysis,”  accepted  by 
Neural  Information  Processing  Systems  (NIPS  95),  December  1995. 

David  Stoutamire  and  Matthew  Kennel,  “Sather  Revisited:  A  High-Performance  Free  Al¬ 
ternative  to  C++,”  Computers  in  Physics,  Vol.9,  No.  5,  Sep/Oct  1995. 

Wawrzynek,  J.,  Asanovic,  K.,  Kingsbury,  B.,  Beck,  J.,  Johnson,  D.,  Morgan,  N.,  “SPERT-II: 
A  Vector  Microprocessor  System  and  its  Application  to  Large  Problems  in  Backpropagation 
Training,”  accepted  by  Neural  Information  Processing  Systems  (NIPS  95),  December  1995. 


